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ABSTRACT 

Part of a series, this preliminary report is on a 
meta-analysis study of successful and unsuccessful transition of 
handicapped youth from secondary education to postsecondary settings. 
The study had four major objectives: (1) to review annual progress 
reports of federally funded (Office of Special Education and 
Rehabilitation) transition projects, (2) to review pertinent 
literature, (3) to perform meta-analysis by project groups, and (4) 
to analyze case studies of successful and unsuccessful job 
placements. The literature review noted a lack of control groups in 
studies on vocational placement of handicapped youth. The funded 
projects were analyzed and grouped based on the Proposal 
Classification Form which considered such features as agency type, 
project objectives, and student age and type as well as the Project 
Characteristics Questionnaire which provided information on agency 
type, student classification, and training and placement practices. 
The initial review of 56 annual progress reports from the projects 
found that only 42% of the projects listed objectives of the original 
proposal in their progress reports and few had met many of their 
objectives. Finally, no clear trends were observable in the 
comparison of five pairs of successful and unsuccessful student 
placements. (DB) 
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The following principles guide our research related to the education and employment of youth and 
adults with specialized education, training, employment, and adjustment needs. 



• individuals have a basic right to be educated and to 
work in the environment that least restricts their right 
to learn and interact with oJhor students and persons 
who are not handicapped. 

• Individuals with varied abilities, social backgrounds, 
aptitudes, and learning styles must have equal 
access and opportunity to engage in education and 
work, and life-long learning. 

• Educational experiences must he planned, delivered, 
and evaluated based upon the unique abilities, social 
backgrounds, and learning styles of the individual. 

• Agencies, organizations, and individuals from a 
broad array of disciplines and professional fields must 
effectively and systematically coordinate their efforts 
to meet individual education and employment needs. 



• Individuals grow and mature throughout their lives 
requiring varving levels and typos of educational and 
employmer i support. 

• The capability of an individual to obtain and hold 
meaningful and productive employment is important 
to the individuals quality of life. 

• Parents, advocates, and friends form a vitally 
important social network that is an instrumental 
aspect of education, transition to employment, and 
continuing employment. 



The Secondary Transition Intervention Effectiveness Institute is funded through the Office of Special 
Education Programs, Office of Special Education and Rehabilitative Services, U.S. Department of 
Education (contract number 300-85-0160). 

Project Officer: Dr. Mel Appell 



For more information on the Transition Institute at Illinois, please contact: 

Dr. Frank R. Rusch, Director 
College of Education 
University of Illinois 
110 Education building 
1310 South Sixth Street 
Champaign, Illinois 61820 
(217) 333-2325 
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CREDITS TO COOPERATING TRANSITION 
EDUCATION PROJECTS 
The Metaanalysis Group of the Transition Institute expressed special 
gratitude to the following projects for taking time from the rigorous 
schedules in order to complete the five questionnaires that we have 
distributed. Those not listed should not necessarily be considered 
unhelpful. Not all projects were eligible for this year's questionnaires, 
and will have to await a later opportunity to participate. Also, the 
person listed with each project is not necessarily the respondent to our 
survey even the directors of the credited project. We try to keep our 
mailing list current, and we apologize for the occasional inoccurances. 

A. Project Identification Questionnaire. 

Mailed March 14, 1986, to identify the projects who might be willing 
to complete a Project Characteristics Questionnaire cr a Case Study on 
both a successfully-placed and an unsuccessfully-placed student. All 
but 9 of the 105 funded projects from B competitions completed and 
returned this questionnaire. 

B. Feasible Variables Checklist. 

Dated January 30, 1986. Mailed to 14 projects, 2 randomly selected 
from each transition education grant competition. 



1. Jack Scott q 



Frank D. Gentile 



SSfC 1 - 1 Ind V stries of Business ' Human Resources Center 

9200 Wisconsin Avenue w nratl - n . a1 D «k,k c, . 

no+hocHa Mn onoiV vocational Rehab. Serv. 

Oethesda, MD 20814 I#u> W iletts Road 

2. John Palmer Albertson, NY 11507 

Human Resources School g. Charles E. Bradford 

SEK*2 n S? ad iisn7 I"fl Assoc. of Machin- 
Albertson, NY 11507 ists & Aer ospace 

•3 M-i+„i~„~ a ,j Workers' Apprenticeship 

r * Employment and Training 

Georgia Retardation Center Department 

AtSen^rf S5tt° n Rd " 1300 Connecticut Ave. N.W 

Athens, GA 30610, Washington, DC 20036 
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4. Gary Gronberg 

Director of Organization and 
Dir. of Employment, Apprent. 
& Training 

North Dai ota Department of Public 

Instruction for Special Education 
Capitol Building 
Bismark, ND 58505 

5. Donna Phillips 

Director of Occupational Education 
Rochester City School District 
131 W. Broad St. 
Rochester, NY 14608 

6. Dr. Robert N. Ianacone 

The George Wasington University 
Office of Sponsored Res. for 
Dept. of Sp. Ed. 
2121 Eye St., N.W. 
Washington, D.C. 20052 

7. Bernard Katz 

New York University 
SEHNAP 

Washington Square 
New York, NY 10003 



10. Dr. Paul Wehman 
Virginia commonwealth 

University 
School of Education 
MCV Box 563 

Richmond, VA 23298-0001 

11. Dr. Dianne Berkell 
C.W. Post Center, Long 

Island Univ. 
Dept. of Special Ed. 
Greenvale, NY 11548 

12. Dr. Jean F. Mooney 
J. Edward Center 
Campus School of Multi- 
handicapped Children 

Commonwealth Avenue 
Chestnut Hill, MA 02167 

13. E. Lowell Harris 
Division of Exceptional 

Children 
NC Dept. of Public 

Instruction 
Education Building 
Raleigh, NY 27611 

14. A. Geneva Quarles 
Crippled Children's 

Hospital . 
2924 Brook Road 
Richmond, VA 23220 



Case Study Checklist. 

Mailed April 21, 1986, to those projects who place students into 
competitive employment. 



Providing Innovative Community-Based Services for Mentally 111 
Persons Exiting Secondary School Programs. 

Dr. Jerry Dincin & Dr. Judith A. Cook, Thresholds. Chicago, IL 
60614 
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The Electronics Industry Enclave Project: A Service Demonstration 
Model for Post-Secondary Individuals with Severe Handicaps. 
Larry E. Rhodes 

University of Oregon, Eugene, OR 97403 

Valley Transitional School Project 
George Drummond 

Valley School, Fisherville, VA 22939 

A Model to Provide Secondary Vocational Preparation of 18 to 21 

Year Old Special Needs Youth 

Vince Svaldi & Wayne Lindskoog 

West Metro Ed Center, Hopkins, MN 55343 

Community-Based Training 
Robert Atkins 

The Rehabilitation Institute, Kansas City, MO 64108 

Training for Effective Transition: A Transition Program for 
Post-Secondary Handicapped Adults Residing in a Rural Area 
Joyce K. Beam 

Charles County Board of Education, Laplata, MD 20646 

MEAL (Model for Employment and Adult Services) 
Dorothy Crawford 

Research and Development Training, Phoenix, AZ 85016 



iii 
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ON-site Employment Training for Handicapped 
Devi Jameson 

Richmond Unified School District, Richmond, CA 94804 
Project Characteristics Questionnaire. 

Mailed May 22, 1986 to those projects who place students into 
competitive employment. 

Human Resources Center Transition of Severely Disabled Youth from 
School to Work: A Demonstration Model 
Roberta Y. Housman 

Voc. Rehab. Services, Albertson, NY 11507 

Illinois Competitive Employment Project: Promoting Non-Sheltered 
Employment Options for Moderately/Severely Handicapped Students 
Tom Lagomarcino 

University of Illinois at Urbana-Champaign, Department of Special 
Education, Champaign, IL 61820 

Providing Innovative Community-Based Services for Mentally 111 
Persons Existing Secondary School Programs 
Dr. Jerry Dincin & Dr. Judith A. Cook 
Thresholds, Chicago, IL 60614 

Helping Hand Rehabilitation Ctr. 
Fred Peters 

Helping Hand Rehabilitation Ctr., Countryside, IL 60525 



Experimental Prevocatlonal Planning Project 
Kay Holjes 

Employment Opportunities, Inc., Durham, NC 27707 

A Model Program of Community College Special Education for Mild 
Mental Retardation 
Daniel Close 

University of Oregon, Special Education and Rehabilitation, 
Eugene, OR 97403 

Valley Transitional School Project 
George Drummond 

Valley School, Fishervllle, VA 22939 

Project REDDY (Real Employment Alternatives for Developmentally 
Disabled Youth) 
Dr. Dianne Berkell 

C.W. Post Ctr., Long Island Univ., Dept. Special Education, 
Greenvale, NY 11548 

Secondary School/Post Training Employment Transition Service 
Demonstration Model Project for Handicapped Students 
Robert A. Stodden 

Univ. of Hawaii, Honolulu, HI 96822 
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A Model to Provide Secondary Vocational Preparation of 18 to 21 
Year Old Special Needs Youth 
V1nce Svaldl 

West Metro Ed. Center, Hopkins, MN 55343 

STEEP (Skills, Training, Evaluation, Education, Placement) 
James M. Caccamo Patrick Mcglnn 

School D1st. of Independence, Independence, MO 64055 

Community-Based Training 
Robert Atkins 

The Rehabilitation Institute, Kansas City, MO 64108 

Training for effective Transition: A Transition Program for 
Post-Secondary Handicapped Adults Residing in a Rural Area. 
Joyce K. Beam, Project Director 

Charles County Board of Education, Laplata, MD 20646 

Project Employment 
Michael Kramer 

Vocational Adult Institute & Workshop, Inc., New York, NY 
10001-2382. 

Post-secondary Transitional Programs 
Jan Rutt 

Vanguard School, Upper School, Paoli, PA 19301 

ii 

vi 



MEAL (Model for Employment and Adult Services) 
Dorothy Crawford 

Research and Development Training, Phoenix, AZ 85016 

Model -Post-Secondary: Improving the Post-Secondary Education and 
EmployablHty of Learning Disabled Students 
Christine Glanopoulos 

Center for Research and Advanced Study, University of Southern 
Main, Portland, ME 04102 

Competitive Employment for Mentally Retarded Young Adults 
Dr. Paul Wehman & Herbert Chermside 

Virginia Commonwealth University, Richmond, VA 23298-0001 

Post-Secondary Education/Rehabilitation Transition for Mildly 

Mentally retarded and Learning Disabled 

Tom E. Bass and Patrick Poplin 

VA Dept. of Ed., Richmond, VA 23216 

Job Training and Try Out: Model Program Designed to I'eet 
Transitional Needs of Out of Work/Out of School Handicapped 
Individuals Ages 18-11 
Dr. George Til son 

The George Washington Univ., Washington, DC 20052 (old address: 
11600 Nevel St., Suite 114, Rockville, MD 20825). 
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To Builii Partnerships for Transitional Services for Handicapped 
Youth 

Thomas Murphy & Joseph Pasanella 

Santa Barbara High School District, Santa Barbara, CA 93103 

PROGRESS (Providing Realistic Opportunities for Gainful 
Rehabilitative Employment Success in Society) 
JoAnn Hankey 

Asso. for Retarded Citizens, Centre County, PA, Inc. , State 
College, PA 16801 

Project LIVE (Learning Independence in Varied Environments) 
Gerry Schwarzentraub 

Stockton Unified School District, Stockton, CA 95202 

Project Interface 84.023N 
Justin Marino 

Arizona State Univ., Tempe, A 85287 

OLTA D00 NAA NISHJI: School -to-Work 
Sherry A. Curley & Elmer J. Guy 

Navajo Vocational Rehab. Program, Window Rock, AZ 96515 

Transition Education Summary Analysis Feasibility Questionnaire 

Mailed February 10, 1986. The 14 projects funded under Competition 

84.158G. 
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Dr. Eugene Edgar 

University of Washington, 103 Miller Building, DQ-12, Seattle, 
WA 98195 

Dr. Martin Agran 

Department of Special Education, Utah State University, Logan, 
UT 84321 

Dr. Carol Weller 

University of Utah, 227 Milton Bennion Hall, Salt Lake City, UT 
84112 

Dr. Thomas Lovitt 

Experimental Education Unit WJ-10, University of Washington, 
Seattle, WA 98195 

Dr. Milton Budoff 

Research Institute for Education, 29 Ware St., Cambridge, MA 
02138 

Dr. Stan Karcz 

University of Wisconsin/Stout, Harvey Hall 201C, Menomonie, WI 
54751 

Dr. Robert Erickson 

University of Wisconsin/Madison, 432 N. Murray, Madison, WI 53706 
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Dr. Steven R. Lyon 

University of Pittsburgh, 5M215 Forbes Quadrangle, Pittsburgh, 
PA 15260 

Tom Kiuwen 

Guallaudet College, 800 Florida Ave., N.E., Washington, DC 20002 
Dr. Robert A. Stodden 

University of Hawaii, 2444 Dole St., Honolulu, HI 9*822 
Dr. Joanna Williams 

Columbia University/Teachers College, New York, NY 10027 
Dr. Edward S. Shapiro 

Lehigh University, Building 2, School of Education, Bethlehem, 
PA 18015 

Dr. Jean Edwards 

Portland State University, P.O. Box 751, Portland, OR 97207 
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Foreword 

The First Annual Monograph of the Meta-Analysis Group at the 
Transition Institute at the University of Illinois summarizes the group's 
first year's objectives and accomplishments. First (Section A) the meta- 
analysis plan (Objective 6.3.2) is presented. Then (Section B) comes a 
brief literature review followed by several presentations of empirical 
information (Sections C thru G) , all of which are ultimately aimed at 
combining data from all the OSERS-funded Transition Education projects that 
place students from school to work settings. Finally, Section H describes 
a plan of research for Year 2. 

L.W.H. 
J.I.H. 



xi 
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META-ANALYSIS MONOGRAPH 
A « Develop a framework for Meta-Analysis 

A meta-analysis is an aggregation of information from a number of 
primary sources. As applied in the present series, it will be an aggrega- 
tion of information about successful and unsuccessful transition from 
secondary education to postsecondary settings-information provided by the 
projects that are being funded by the Office of Special Education and 
Rehabilitative Services (OSERS). 

The Meta-Analysis Plan (Objective ft .3.?) 

The meta-analysis plan was prepared on schedule for the March 15 
monthly report. This plan appears as Attachment Al. The schedule of 
activities from this plan shows that the group has assumed four major 
responsibilities: 

1. review annual progress reports (Objective 6.2.1) 

2. review pertinent literature (Objective 6.2.2) 

3. perform meta-analysis by project groups (Objective 6.2.4) 

4. analyze case studies of successful and unsuccessful job place- 
ments (Objective 6.2.11) 

Preliminary reports on these four appear in the present report. The 
schedule of completion appears in Attachment Al. 
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Attachment Al 
Transition Education Meta-analysis Plan 
March 10, 1986 
Laird W. Heal 
Janell I. Haney 

The plan for the Meta-analysis of the Transition Institute consists of an 
elaboration of the management plan from the original (1985) proposal, which 
is shown belcw on attachment A2. The expansion involves Objectives 6.2.11 
and 6.2.12. These objectives are new and were not mentioned in the origi- 
nal proposal. 

Objective 6.2.11, "Case Studies of Successful and Unsuccessful Placements, " 
is a project to ask the OSERS service projects to nominate successful and 
unsuccessful placements of students from high school training programs into 
the competitive employment marketplace. Each project will be asked to 
nominate a typical successful case and a typical unsuccessful case. The 
reasons for success or failure will be noted in case study that will be 
completed according to a structured format. This format will yield both 
objective and subjective information about these cases. During the early 
summer (of 1986) the Meta-analysis project staff will complete a content 
analysis of these data and write a report highlighting the variables that 
discriminate between successful and unsuccessful transition education 
placement. 

Objective 6.2.12, "Compare OSERS Research Projects from Competition 
84.158G" will compare the research projects in transition education that 
were funded by OSERS in 1984. Only 14 of these projects were funded, and 

■18 
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the comparisons will probably be tentative at this early date, but the 
framework for classifying and collecting data from these projects will at 
least be put in place, and a more complete research plan will be put in 
place for FY 1986-87. 

The "Meta-analysis Plan 6.2 Expansion Timetable" is attached. This 
timetable shows the activity schedule for the 10 objectives listed in the 
Transition Institute contract proposal and the two (6.2.11 and 6.2.12) 
added above. 
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Meta-Analysis Plan 6,2 -- Expansion Timetable 



fin 

FRir 



X 
X 



1985 1986 
Oct Nov Dec Jan Feb Mar Apr May June Jul Aug 

6.2.2 Literature Review of Previous Placement Research 

1. Collect and read literature y x * v v v 

2. Write literature review (6.2,2) x 

3. Write report on meta-analysis procedures (6.2.9) X 

4. Submit article on meta-analysis procedures ' x 
(6.2,10) 

x 

6.2.4 Compare OSERS Research Projects by Project Groups 

1. Determine classification and dependent variables x x x x x « 

2. Set data from those projects that can be 
compared (6.2.4) 

3. Analyze data (6.2.4) 

4. Write draft report (6.2.5) 

5. Review of draft report by Institute Advisory 
Committee, RTI Staff, and federal project 
officer (6.2.6) 

6. Hake needed revisions (6.2.7) 

7. Submit/disseminate final report (6.2.8) 

6.2.11 Case Studies of Successful and Unsuccessful Placements 

1. Identify projects that place students 

2. Hake up case study topics form 

3. Recruit 1 typical success and 1 typical failure 
from each 

4. Get Data 

5. Do preliminary content analysis of data 

6. Analyze preliminary results 

7. Write report 

6.2.12 Compare OSERS Research Projects (84.158G) 

1. Determine classification and dependent 
variables 

2. Prepare plan for meta-analysis of research 
projects 



x x x x x 
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Attachment A2 



MANAGEMENT PLAN 



OBJECTIVE (6.2): Conduct meta-analyses of model programs 
to delineate objectives 

OBJECTIVE MANAGER: Dr, Laird Heal 



PROJECT: Secondary/Transition Institute 
TASK 6: Evaluation 



TofT 



ACTIVITY 



6.2.1 Collect and review the initial annual project progress/avaluatiun 
reports 

6.2.2 Obtain and review other pertinent literature (Task 1.0) 

6.2.3 Develop a framework for the meta-analysis 

6.2.4 Perform the meta-analyses by project groups (5) 

6.2.5 Produce draft meta-analysis reports 

6.2.6 Submit tho draft report(s) for review by Institute Advisory 
Committee, RTI staff, and federal project officer 

6.2.7 Make needed revisions 

6.2.8 Submit/disseminate final meta-analysis reports 

6.2.9 Review and evaluate meta-analysis procedures 

6.2.10 Develop and submit articles on meta-analysis procedures/ 
findings 



~ 

LH s Dr. Laird Heal 

GRA 7 - Graduate Research Assistant 

AP • Dr. A. Phelps 

IAC • Institute Advisory Committee 

RTI - Research Triangle Institute staff 

PO • Project Officer 
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PRODUCT 



DATE OF 
INITIATION 



DATE OF 
COMPLETION 



initial review 8-21-85 
notes 



Completed 
framework 



8-21-85 
1-1-86 



Final reports 



Journal 
articles 



on -going 

on-ging 
3-1-86 



PERSONNEL 
INVOLVEMENT 



LH, GRA 7 

LH, GRA 7, AP 
LH, AP, GRA 7 



4-15-86 


5-15-86 


LH, GRA 7 


5-1-86 


6-1-86 


LH, GRA 7 


6-1-86 


6-15-86 


LH, IAC, RTI, 
PO 


6-10-86 


6-30-86 


LH, GRA 7 


8-1-86 


2-20-86 


LH, AP 


7-1-86 


8-20-86 


LH, GRA 7, AP 


6-15-86 


8-20-86 


LH, GRA 7 
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B. Obtain and Review Relevant Literature 

Meta-analysis is the name that has been applied to a recent strategy 
of combining research data from a number of studies. This strategy has 
been most noticeably identified with Glass, McGaw, and Smith (1981), but a 
number of others have made important contributions to this methodology and 
its rationale. Bangert-Drowns (1984) has provided an especially insightful 
historical review, and Hedges (1985) has presented advanced statistical 
procedures. 

The meta-analysis strategy for combining data from a number of studies 
employs the following procedure: 

1- Locate all studies that ask the same question regarding the rela- 
tionship between an independent variable (an intervention) and a 
dependent variable (an outcome measure) 

2. Convert every comparison in every study that contrasts an 
experimental group with a control group to a test statistic; this is 
usually the effect size , which is defined as the difference between 
the two groups 1 means divided by the standard deviation of the control 
group 

3. Calculate the average effect size for all comparisons in all 
studies; This statistic, like any z-score, has a theoretical mean of 
zero and a practical range from -3.0 to +3.0 

4. Interpret the results— a positive effect size means that the 
experimental groups had higher average outcome scores than the control 
groups; however, ordinarily an effect size must reach a quarter of a 
standard deviation (+.25) in order to be considered practically 
important. 

24 
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It should not be surprising that this meta-analysis procedure has 
become popular to combine the effects of special education interventions 
since many "efficacy studies" undertaken to evaluate them have compared 
treated and untreated groups. Kavale and his associates have been 
particularly active. In their review of meta-analyses in special educa- 
tion, Kavale and Glass (1982) discussed meta-analyses of five different 
literatures in special education: a) Special class vs. regular class place- 
ment, b) psycholinguistic training using the Illinois Test of 
Psycholinguistic Ability as the outcome measure and training guide, 
c) perceptual -motor training, d) stimulant drugs as treatment for 
hyperactive behavior, and e) diet as treatment for hyperactive behavior. 
Effect sizes were positive (favorable toward the intervention) for nearly 
all analyses, approaching 1.0 for drug treatment of hyperactivity, and 
approaching 0.0 for diet control of hyperactivity. However, the average 
effect size for academic outcome after special placement was -.14 for 
educably mentally retarded students and -.34 for "slow learners." Special 
class outcomes for social variables were more positive, the average effect 
size being .29. In addition to the studies reviewed by Kavale and Glass, a 
number of recent meta-analyses have investigated the effectiveness of early 
intervention. Lazar and Darlington (1982), Snyder & Sheehan (1983) and 
White and Casto (1985), among others, reported effect sizes ranging from 
+ .3 to +.5 for the immediate effects of early intervention on the school 
readiness of handicapped and disadvantaged children. 

Conspicuously absent from this literature as a meta-analysis comparing 
vocational placement strategies for handicapped youth. Unfortunately, very 
few studies of vocational placement of handicapped youth have employed 
control groups, making it impossible to calculate a conventional effect 

25 
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size. Nevertheless, some studies have been completed, and they have been 
expertly reviewed by Cobb (1972), Conley (1973), and Kolstoe (1975). The 
stunning result of these studies is that youths who are followed after 
graduation from special education programs have an employment history that 
is very similar to their nonhandicapped peers of a comparable socioeconomic 
status. Cobb remarked that employment success for handicapped workers 
often comes after many unsuccessful attempts. Conley' s analysis was more 
refined, in that he considered the ages of the cohorts that were studied by 
the various investigators. He found that the employment rate of 
"graduates" of special education programs was less than 10 percent below 
that of their same-age peers. 

The studies in Cobb's (1972) and Coney's (1973) reviews were based 
primarily on mildly mentally retarded y ir. i n one classic study by 
Saenger (1957) followed 520 moderately men. il' retarded (IQ range: 40-50) 
"graduates" of New York City's special education programs over the 25-year 
period from 1929 to 1956. He found that fully 37 percent of these 
graduates were gainfully employed. Clearly, the past practices have 
resulted in the gainful employment of ^ost of the graduates of special 
education programs, leaving open to question the wisdom of spending extra- 
ordinary resources to do better. 

Kolstoe (1975) in reviewing the reasons for unsuccessful job place- 
ment, concluded that most placement failures result from social 
incompetence, not skill deficiencies. Kolstoe emphasized the importance of 
providing environmental and training supports for the handicapped 
individual at the job site. 

Following the instatement of the law requiring public educational 
services for all handicapped youths through age 21 (P.L. 94-142), increas- 
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ing concern has been given to the transition of moderately and severely 
mentally retarded youths from school to adult living. While many services 
have been forthcoming, the evaluation of these services has been almost 
nil. The research investigating the factors that might facilitate or 
impede successful placement has used the most primitive research designs. 
Typically retrospective, one-shot studies have been done, with investi- 
gator's asking practitioners (e.g., Hasazi , Gordon, & Roe, 1985) or clients 
(e.g., Kernan & Koegel , 1985) what factors influenced adjustment to voca- 
tions and other activities. 

Unfortunately the comparisons made of the projects that have been 
funded under the recent transition education priority of the Office of 
Special Education and Rehabilitative Services (OSERS) must be made using 
the same substandard evaluation designs that have become standard in recent 
years. Definitive answers are unlikely. It is with this misgiving that 
the "meta-analyses" performed by the Transition Institute have been under- 
taken. These analyses have necessarily departed from the model that was 
described above, since comparison groups are nonexistent. In order to 
combine the information from the many projects funded by OSERS as transi- 
tion education demonstrations, several strategies have been adopted (see 
the Management Plan, Section A, above). 

While these analyses are undermined by the practical limits of field 
research, it is expected that they will reveal much new information 
regarding the environmental conditions, program features, and personal 
characteristics that optimize the success of vocational placements for 
handicapped youth who are in transition from school to the community. 



27 



0'\ 

ERIC 



10 



C. Classify Projects Using the Proposal Classification Form 

A major activity of Task 6.2 was to perform a meta-analysis by project 
groups. A "proposal classification form" was initially developed for the 
purpose of grouping projects (i.e., establishing dependent and independent 
variables) for this meta-analysis. Section C describes the classification 
form, the frequency with which projects fell into the various categories on 
the form, and the benefits that have been derived by this sub-component of 
objective 6.2.4. 

Method 

The "proposal classification form" categorized projects in terms of 
their agency type, project objectives, number of clients, client age, 
client type, community type, training setting, and timing of training. As 
implied by the title of the form, projects were categorized solely on the 
basis of information given in the proposal. 

A blank classification form follows as Attachment CI. The form 
consisted of several sections, which were completed by project staff. 

1. Information regarding project identification (i.e., title, 
proposal numbers, city, county, and state, agency type, number of clients, 
and community type) was obtained directly from the proposal cover sheet. 

2. Checklist of project objectives explicitly stated in the 
objective section of the proposal. 

3. Client age (21 or younger, over 21), client type (e.g., mild 
mental retardation, learning disability), location of training (e.g., 
school, workshop), and timing of training (i.e., pre- or post-placement)— 
information obtained primarily from the method section of the proposal. 

Forms were completed by seven evaluation program staff members. 
Proposals were selected unsystematically for completion. A total of 20 
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proposals, unsystematically selected, were rated by two different indi- 
viduals. Agreement statistics were based on these 20. 

Results 

Frequencies for the various categories of The Proposal Classification 
Form are presented in Table CI. 

Agency type. The grantee for most projects (41 of the 105 examined 
proposals) was found to be a university. Other grantee agencies were found 
to include community not-for-profit agencies, state education or rehabilita- 
tion facilities, schools, and one community college. 

Project objectives . The three most frequently cited objectives 
(over 50 of the projects for each) were information dissemination, voca- 
tional training of students, and interagency planning and/or coordination. 
Thirty or more projects listed student placement, student pre-placement 
assessment, nonvocational training of students, and product development as 
objectives. Other reported objectives included follow-up support, 
technical assistance, student referral, and family training. 

Student age . A total of 97 projects reported serving students that 
were 21 or younger. A total of 46 reported serving students over 21 years 
of age. 

Student type . The most frequently reported student classifications 
were learning disability (42) and either moderate/severe mental retardation 
(27) or mental retardation (27). Other classifications included emotional 
handicap (17), physical handicap (17), sensory handicap (12), multiple 
handicap (12), communication disorder (2), and "at risk" (2). 



Insert Table CI about here 



Table CI 

Transition Institute Proposal Classification 



12 



* 
I. 



Competition (CFDA#) 



84.023D 
84.078B 
84.128A 
84.158B 



12 
15 
5 

11 



84.0236 
84.078C 
84.158A 
85.158C 



15 
14 
16 
17 



TOTAL 



8 COMPETITIONS 



105 PROPOSALS 



'il. Agency Type (0=does not apply, l=grantee, 2=secondary) 



1. 
2. 
3. 
4. 



5. 
6. 



7. 



9. 

b 10. 
11. 



school 

community college 
university 
state ed. or 
rehabilitation 
facility 

community workshop 

community ed. 

or rehabilitation 

facility 

other community 

not-for-profit 

agency 

profit-making 
agency 

several cooperating 
agencies 
other 
unknown 



0 


1 


2 


90 


12 


3 


104 


1 


0 


60 


41 


4 


90 


13 


2 


105 


0 


0 


100 


2 


3 


89 


14 


2 


105 


0 


0 


98 


0 


7 


79 


21 




105 


0 
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Table CI (continued) 

III. Project Objectives enumerated in proposal 

("Choose all that apply.") ("0" = No; "1" = yes) 







0 


1 


1. 


student referral 


88 


15 


2. 


student pre-placement 






assessment 


64 


33 


3. 


student placement 


68 


36 


4. 


student training-voc. 


39 


62 


5. 


student training- 








non-vocational 


70 


31 


6. 


technical assistance 


79 


22 


7. 


inter-agency planning 








and/or coordination 


43 


55 



5. community workshop 105 0 

8. product development 68 30 

9. follow-up support 77 25 

10. family training 85 13 

11. info, dissemination 33 63 

12. other 50 54 

13. unknown 104 i 

IV. Student Age (0=Yes; l=No) 
("Choose all that apply") 



1. 21 or younger 8 97 

2. over 21 59 46 



V. Student Type (0=Yes; l=No) 
("Choose all that apply") 







0 


1 


1. 


moderately and severely 


78 


27 


2. 


mentally retarded 


78 


27 


3. 


learning disabled 


63 


42 


4. 


emotionally handicapped 


88 


17 


5. 


"at risk" 


103 


2 


6. 


physically handicapped 


88 


17 


7. 


sensorily handicapped 


93 


12 


8. 


communication disordered 


103 


2 


9. 


multiply handicapped 


93 


12 


10. 


all or most types 


94 


11 


11. 


other 


60 


45 


12. 


unknown 


104 


1 



> .7 
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Table CI (continued) 

VI. Type of Community or Area Served (Choose one only) 
("Choose one") 



1. metropolitan 27 

2. suburb 6 

3. small town (5000-100000) 1 

4. county 12 

5. region 17 

6. statewide 22 

7. other 13 

8. unknown 5 

VII. Location of Training 

("Choose all that apply 11 ) 



1. school 58 

2. workshop 9 

3. OJT 46 

4. other 14 

5. no training 21 

6. unknown 2 

VIII. Timing and Training 

("Choose all that apply") 

1. preplacement 55 

2. postplacement 47 

3. no training 20 

4. other 11 

5. unknown 3 



a Note: Unless otherwise indicated, proposals were counted under all 
categories that applied to them, so the count in each of the VIII 
classification areas usually exceeds 105. 

^These 21 were primarily four-year colleges and non-profit community 
agencies. 
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Area served . Projects were found to serve a variety of areas. A 
number reported serving entire states (22), regions (17), or counties 
(17). A total of 27 stated that they serve metropolitan areas, 6 stated 
that they served suburban areas, and 1 stated that it served a small town. 

Location of training. Most training sites were found to be schools 
(58) or on-the-job sites (46). Nine projects were located in workshops. 

Timing of training. About half the projects had training before 
placement (55) and about half (47) had training after placement. Twenty 
projects appeared to have no training. 

Reliability . 

Twenty proposals were randomly selected for rating by two raters. All 
items within each category except "number of clients" were scored 
dichotomously as being rated present/absent by each rater. Interrater 
agreement on the presence/absence of each item was then assessed. For 
"number of clients," agreement on the specific number of clients impacted 
was assessed. Both percent agreements and Kappas were then calculated to 
determine inter-rater reliability. Percent agreement is calculated by 
dividing agreements by the sum of the agreements and disagreements. Kappa 
is calculated by the formula, K=(A-E)/(N-E) , where A = agreements, 
E = expected agreements by chance, and N = number of items. Kappa, 
described by Cohen (1960), corrects the chance agreement among raters. 

Although percent agreements were often acceptable (30 of 45 items had 
percent agreements of 80% or higher), Kappas were acceptable only about 
half of the time (22 of 45 items had Kappas of .60 or more). However, 
ratings for some categories were more reliable than ratings for others (see 
Table C2). 
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Insert Table C2 about here 



There was a Kappa of 1.0 (total agreement) for number of clients and 
.56 (approaching acceptability) for client age. However, it may be that 
some disagreements were avoided in these areas as they were often left 
uncoded (perhaps when the information was not readily available on the 
cover sheet or required inference). 

Type of agency and client disability also had fairly high inter-rater 
reliability, with all but a few items at Kappas of .60 or above. There was 
low agreement regarding categorization of agencies as "community education 11 
or "rehabilitation 11 facilities and "other community nonprofit" as well as 
categorization of disability types as "at-risk," "sensory handicap, 11 
"multiple handicap," or "all or most types." The reliability for objective 
type was acceptable in about half of the cases. Low agreement was found 
for client placement, vocational training, nonvocational training, 
technical assistance, and information dissemination. 

There was poor agreement regarding the assignment of projects to the 
categories within "Area served," "Location of training," and "timing of 
training." 

In summary, reliability was generally acceptable for "number of 
clients," "client age," "type of agency," and "client disability." 
Reliability for "objective type" was often acceptable. Reliability for 
both "area served" and "location of training" were each acceptable for only 
one item; reliability for timing of training was unacceptable for all items. 
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Table C.2. 

Reliability of Proposal Classif icatioi, Fonn Ratings. 



Category 



Item | Description 



Number 
of Cases 


Pprrpirf 
An rppmpnt 




19 


95 


Nappd 

.85 


19 


100 


1.00 


19 


84 


.66 


19 


95 


77 


19 


100 


1 nn 


19 


95 


.00* 


19 


84 


-.08* 


19 


inn 


l nn 
x • UU 


19 


inn 

X uu 


1 nn 
X .UU 


19 


inn 

X \J\J 


1 nn 
x . uu 


19 




• DD 


19 


68 


.33* 


19 


68 


or* 


19 


68 


.20* 


19 


75 


-.14* 


19 


84 


.68 


19 


89 


.76 


19 


84 


.62 


19 


95 


.64 


19 


58 


.16* 


15 


100 


1.00 



Type of 
Agency 



Objective 



Number of 
Clients 



Dl School 

D2 Community college 

D3 University 

D4 State education or 

rehabilitation agency 
D5 Community workshop 

D6 Community education or 

rehabilitation agency 
D7 Other community nonprofit 

D8 Profit-making agency 

D9 Several cooperating agencies 

El Client referral 

E2 Client pre-placement 

assessment 

E3 Client placement 

E4 Client training-vocational 

E5 Client training-nonvocational 

E6 Technical assistance 

E7 Interagency planning/ 

coordination 

E8 Product Development 

E9 Follow-up support 

E10 Family training 

Ell Information dissemination 



15 



80 



A 9e Gl 21 or younger vs. all ages 15 80 57 

(There were no projects with only students over 21 among those randomly selected 
for reliability assessment.) 

Client HI Moderate and severe mental 

disability retardation 

type H2 Mild mental retardation 

H3 Learning disability 

H4 Emotional handicap 

H5 "at risk" 

H6 Physical handicap 

H7 Sensory handicap 

H8 Communication disorder 

H9 Multiple handicap 

H10 All or most types 



19 


84 


.64 


19 


85 


.65 


19 


84 


.67 


19 


95 


.83 


19 


95 


0.00* 


19 


89 


.68 


19 


95 


0.00* 


19 


100 


1.00 


19 


84 


.31* 


19 


79 


.37* 
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Category 


Item # 


Description 


Number 
of Cases 


Percent 
Agreement 


Kappa 


Area 


11 


Metropolitan area 


18 


61 


.16* 


served 


12 


Suburb 


18 


89 


0.00* 




T O 
13 


Small town 


18 


100 


1.00 




14 


County 


18 


78 


.20* 




15 


Region 


18 


72 


-.15* 




16 


Statewide 


18 


72 


-.10* 


Location 


Jl 


School 


18 


67 


.31* 


of training 


J2 


Workshop 


18 


100 


1.00 




J3 


On-the-job training 


18 


95 


-.44* 


Timing of 


Kl 


Preplacement 


19 


53 


.03* 


training 


K2 


Postplacement 


19 


58 


.16 




K3 


No training 


19 


79 


0.00* 



*=unacceptable Kappa of .60 or less. 



36 



ERIC 



19 



Discussion 

In summary, a variety of objectives have been established in a variety 
of settings and with a variety of students. Training, inter-agency 
coordination, and information dissemination are frequent goals. A large 
percentage of the project grantees are universities. Most often students 
are 21 or younger with mental retardation or learning disabilities. 
Training sites are fairly equally split between schools and actual job 
sites with some workshops. Areas of impact are most often metropolitan. 
However, the primary characteristic of the projects is diversity. 

Although subsequent correspondence with the various project directors 
has indicated that the proposals themselves do not represent the most 
up-to-date and accurate information regarding the projects, the proposal 
classification forms have served at least two purposes: 

1. They have provided an orientation to the projects. For example, 
they have assisted in establishing appropriate questions for subsequent 
questionnaires. 

2. They provided the most comprehensive initial picture of the 
proposed projects. They thus give an account of initial intentions that 
may be later compared with actual actions and outcomes. 

3. They provide a classification screen, so that questionnaire 
mailings can be limited to the projects that are appropriately addressed by 
them. 
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Attachment CI. 



Transition Institute Proposal Classification Form 

October 14, 1986 

Directions: 



A. 
B. 
C. 
D. 

0 
0 
0 
0 



SSJSl^S-?! 1 b1anks and c1rcle the numbers that Indicate vour choir** 

Title of proposal: ■ 
Competition (CFDA) T 
City 



UStKS Proposal 



— * , County 

Agency type <Q»does not apply, l«grantee7 



ti Proposal 
State 



T 



0 1 2 



0 
0 
0 
0 

E. 



2«secondary) 

1. school 

2. community college 

3. university 

4. state education or rehabili- 
tation facility 

5. community workshop 

6. • community education or 

rehabH1t§t1on facility 

7. other community not-for- 
profit agency 

8. profit-making agency 

9. several cooperating agencies 

10. other (specify) 

11. unknown " 



Client type (choose al I that apply):" 



Project objectives (rate 0, 1, or 2)* 
client referral 

client pre-placement assessment 
client placement 
client training - vocational 
client training - non-vocational 
technical assistance and 
Inter-agency planning and/or 
coordination 
product development 
follow-up support 
family training 
Information dissemination 
other (specify) 
unknown 



F. Number of clients 



0 


1 


2 


1. 


0 


1 


2 


2. 


0 


1 


2 


3. 


0 


1 


2 


4. 


0 


1 


2 


5. 


0 


1 


2 


6. 


0 


1 


2 


7. 


0 


1 


2 


8. 


0 


1 


2 


9. 


0 


1 


2 


10. 


0 


1 


2 


11. 


0 


1 


2 


12. 








13. 



F. Client age (choose all that apply): 

1. 21 or younger 

2. over 21 

*Use the following code: 0«does not apply; 
I=expl1c1t objective 1n proposal; 2*clearly 
Implied objective 1n proposal. 



1 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 
10. 
11. 
12. 



moderately and severely 
mentally retarded 
mildly mental retarded 
learning disabled 
emotionally handicapped 
"at risk" 

physically handicapped 
sensorlly handicapped 
communication disordered 
multiply handicapped 
all or most types 

other (specify) 

unknown 



Type of community or area served 
(Choose one only): 
1* metropolitan area 

2. suburb 

3. small town (5000-100000) 

4. county 

5. region 

6. statewide 

7. other (specify) 

8. unknown 



Location of training (Choose all 
that apply): 



1. 
2. 
3. 
4. 
5. 
6. 



school 
workshop 

OJT (on the job training) 

other (specify) 

no training — - 
unknown 



Timing of training (Choose all 
that apply 
!• preplacement 

2. postplacement 

3. no training 

4. other (specify) 

5. unknown 
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D - Classify Projects Using The Project Characteristics Questionnaire 

Successful placement 1s apparently affected by many factors, including 
some associated with training and placement agencies. One approach being 
taken to Identify the variables associated with successful placement in 
employment 1s to ask agencies to describe their training and placement 
procedures and give objective data regarding their success in placing 
handicapped youths into competitive employment. To Implement this 
approach, the meta-analysis staff developed the Project Characteristics 
Questionnaire. 

Method 

In May, 1986, Project Characteristics Questionnaires were mailed to 
the 50 of the 103 projects on the Transition Institute's roster of transi- 
tion service projects who had indicated on an earlier questionnaire that 
they actually place students into competitive employment. The Project 
Characteristics Questionnaire and its instructions appear as an attachment 
in April, 1986, monthly report and as appendix Dl below. Questions A, B, 
and C asked for identifying information. Question D focuses on agency 
type. Respondents define their agencies by selecting "all that apply" from 
a list of 15 agency types. Question E asks the respondent to check which, 
if any, of 20 objectives are a) written in project manuals or the project's 
grant proposal or b) practiced but not written down. Question F asks for 
the classification of students by type and severity of handicap. Question 
G asks the respondent to check which, if any, of 11 instructional 
approaches is used by the project. Question H asks the respondent to check 
which type of community is served by the project. Question I thru L asks 
for features of each project's training and placement practices and 
results. Question M thru 0 asks the project to identify characteristics of 
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Itself and Its setting that facilitated or Impeded successful training and 
placement of students. 

By the time of the writing of this report 13 of the 50 had been 
returned. The report of this objective (6.2.4B) 1s based on these 13. 

Results 

The responses to the Project Characteristics Questionnaire are shown 
in Tables Dl through f\ Table Dl Indicates that 6 were public or private 
secondary schools, 5 were universities or research Institutes, and 3 were 
other community education or rehabilitation facilities. 



Insert Table Dl about here 



Six projects served metropolitan areas. One was in a rural area. 
Three serve an entire county and two serve a region within a state. One 
project served a rural area and one a suburb. None of the projects served 
small towns. None were statewide, nationwide, or served a region of the 
country. 

Table D2 shows the primary objectives of the respondent projects. A 
list of 20 objectives was provided, and respondents were asked to indicate 
whether they were committed to each objective in writing. All 13 
respondents indicated that student referral and information dissemination 
were written objectives for their projects. Twelve projects had written 
objectives for student preplacement assessment, student training-vocational 
preparation, follow-up support and/or training, interagency planning and 



Insert Table D2 about here 
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Table Dl 

Type of Respondent Agency 



Item Agency Type 



Grantee 



Directly involved 
but as secondary 
agency 



1 public school 5 

2 private school 1 

3 community or junior college 0 

4 university or four year college 2 

5 JTPA Service Delivery Area Agent 1 

6 state residential education or 
rehabilitation facility 0 

7 community workshop o 

8. community education or 
rehabilitation facility 3 

9. profit-making agency o 

10. several cooperating agencies 0 

11. research institute 3 

12. city or county government 0 

13. state agency 0 

14. other* 3 



4 
3 
5 
3 
6 

5 
6 

7 
3 
7 
0 
6 
7 
2 



♦Three of these were community not-for-profit agencies (cf. category 8) 
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Table D2 

Project Objectives (Activities) of the Transition Education 
Projects that Place Students (N=13) 

Docu- 

Does mented Not 

Not Objec- Docu- 

Apply tive mented Activity 



0 


13 


0 


student referral 


0 


12 


1 


student preplacement assessment 


0 


11 


2 


student placement 


1 


12 


0 


student training - vocational preparation 


1 


11 


1 


student training - novocational (e.g., academic, 
personal-social, daily living skills) 








2 


8 


3 


student counseling 


0 


12 


1 


follow-up support and/or training 


2 


7 


4 


family training or counseling 


3 


8 


2 


case management 


0 


13 


0 


information dissemination 


2 


6 


5 


technical assistance 


3 


5 


5 


inter-agency planning and/or coordination: 
state level 


0 


12 


1 


inter-agency planning and/or coordination: 
local level 


3 


8 


2 


product development 


3 


5 


5 


assessment of transition education system 


0 


7 


6 


evaluation of transition education outcomes 


1 


12 


0 


development of a model that can be replicated 


10 


3 


0 


replication of an existing transition education 
model 


6 


6 


1 


training professionals, paraprofessionals, or 
prospective educators for transition education 








11 


1 


1 


other (specify) 


13 


0 


0 


unknown 
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coordination at the local level, and development of a model that could be 
replicated. Eleven projects had written objectives of student placement 
and student training-nonvocational . All other listed objectives were 
selected by at least one of the 13 respondents, indicating the rich varia- 
tion in the ideologies and activities of these projects. This richness 
will facilitate the detection of especially salient project characteristics 
that facilitate successful student job placement. 

The 13 respondent projects indicated that they served 223 mentally 
retarded students, 308 learning disabled students, 128 emotionally handi- 
capped persons, nine physically handicapped students, three students with 
visual impairments, and six students with hearing impairments. Fifty-five 
students were reported to be multiply handicapped, n-;ne of whom were non- 
ambulatory. These figures represent some double counting of the students 
served. 

Respondents indicated that the most popular instructional approaches 
utilized for students were individualized instruction and modeling and 
imitation. Eleven projects used these approaches. Ten projects used task 
analysis, group instruction, and on-site training. Nine projects used 
simulations before on-site training. Six projects used peer tutoring, four 
use lectures, and one used "reciprocal teaching." 

When they were asked about the location of students' training, eleven 
agencies reported that training took place on the job at a minimum wage. 
Nine reported that training took place in the community. Eight reported 
training in school. Four reported training at a workshop/rehabilitation 
center. Five reported "other" training options. 

Six of the respondent projects provided preplacement training; ten 
provided postplacement (on the job) training; and one provided "no 
training." 
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Ten of the projects provided information on the number of students in 
various employment categories in 1985. One hundred eighty-three students 
were reported to be employed full- or part-time in jobs that paid above 
minimum wage. Five of these jobs were part-time subsidized jobs. One 
hundred fifty-four jobs were part-time nonsubsidized positions. Twenty- 
four were full-time jobs that were not subsidized. Nineteen students were 
reported to have part-time jobs that paid below minimum wage; thirteen of 
these jobs were subsidized and six were not. Three projects reported the 
number of students—a total of 30~that were npt employed. 



Insert Table D3 about here 



When asked about constraints on transition education, the most common, 
marked in 8 proposals, was limited funding. The next most commonly 
reported constraint was "parents interfere with transition. 11 Only five 
respondents indicated that transition education is constrained by the fact 
that "jobs are hard to find." Five reported "other" constraints: three 
indicated that transportation was a problem; the other two were lack of 
school administrative support and student attitudes. Four projects 
reported that "1 aws constrain pi acement. " Four indicate that "teachers 
need more skill s . " Three respondents reported that "communi ty attitudes 
interfere with placement." Two projects were constrained because housing 
was hard to find. Only one was constrained because employers and staff 
feared liability suits. 
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Table D3 



Number of Students Emplo yed Part or Full-time. Above or Below Mi nimum Wage, 
whose Employers Receive a Subsidy or not - a - L 



Subsidized Nonsubsidized 
Part-time Full time Part-time Full time Total 



Above Minimum 
Wage 


5 


0 


154 


24 


183 


Below Minimum 
Wage 


13 


0 


6 


0 


19 



Total 18 0 160 24 202 
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Conclusion 

This summary of response to the Project Characteristics Questionnaire 
is only preliminary. The number of responses, 13, is too low to allow 
conclusive statements and generalizations to be made. However, the 
responses already received indicate some refinements that should be made in 
the questionnaire. In addition, the patterns of responses suggest trends 
that it will be interesting to investigate. For instance, will other 
projects also report parent interference as a constraint to transition 
education? When more questionnaires have been returned, such questions can 
be explored. Also the relationships among reported project characteristics 
will be analyzed, and the characteristics that are associated with 
successful placement may be identified. 
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Attachment Dl 
Project Characteristics Questionnaire 
April 28, 1986 
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DRAFT: PLEASE COMMENT 



Reviewer Review Date 



Secondary and Transition Institute (TI) Project 
Characteri sti cs Questionnai re 
Laird W. Heal 
April 28, 1986 



Directions: Complete all blanks and circle the numbers that indicate 
your choices. Note that some lettered items permit only 
one choice and others permit several. 

A. Title of proposal : 

B. Competition (CFDA) # OSERS Proposal # 

TI Proposal # 

C. City County State Zip 

D. Agency type (0 = does not apply, 1 = grantee, 2 = directly involved 

but secondary agency) (Circle all that apply.) 



0 ] 


I 2 


1. 


public school 


0 ] 


I 2 


2. 


private school 


0 ] 


I 2 


3. 


community or junior college 


0 ] 


I 2 


4. 


university or four-year college 


0 ] 


I 2 


5. 


JTPA Service Delivery Area agent 


0 ] 


I 2 


6. 


state residential education or rehabilitation 








facility (institution). 


0 ] 


L 2 


7. 


community workshop 


0 ] 


I 2 


8. 


community education or rehabilitation facility 


0 ] 


I 2 


9. 


profit-making agency 


0 ] 


I 2 


10. 


several cooperating agencies 


0 ] 


I 2 


11. 


research institute 


0 ] 


I 2 


12. 


city or county government 


0 ] 


I 2 


13. 


state agency(s) (specify) 


0 ] 


I 2 


14. 


other (specify) 


0 ] 


I 2 


15. 


unknown 
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nl?nn e t^ d j^ e ^ P^ 4 . objectives (i.e., primary activities) 
using the following codes (circle all that apply): 

0 = does not apply 

1 = an objective of the project that has been written down in the 

project proposal or in project manuals 

? - * project objective that is practiced but has not been written 
own 

2 1. student referral 
2 2. student preplacement assessment 
2 3. student placement 

2 4. student training - vocational preparation 
2 5. student training - nonvocational (e.g., academic, 

personal-social, daily living skills) 
2 6. student counseling 
2 7. follow-up support and/or training 
2 8. family training or counseling 
2 9. case management 
2 10. information dissemination 
2 11. technical assistance 

2 12. inter-agency planning and/or coordination: state 
level 

2 13. inter-agency planning and/or coordination: local 
level 



2 14. product development 

2 15. assessment of transition education system 
2 16. evaluation of transition education outcomes 
2 17. development of a model that can be replicated 
2 18. replication of an existing transition education 
model 

2 19. training professionals, paraprofessionals, or 
prospective educators for transition education 

2 20. other (specify) 

2 21. unknown 



Comments: 
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F. Please enter the number of students directly served by your project 
during the current project year in each of the following disability 
categories. Count each student exactly once. 

a) moderately b) mildly c) at risk 

& severely disabled 
disabled 



AMBULATORY STUDENTS 

1. mentally 
retarded 



2. learning 
disabled 



3. emotionally 
handicapped 



4. physically 

handicapped 



5. visually 
impaired 



6. auditorily 
impaired 



7. communi cation 
disordered 



8. multiply 

handicapped 



9. nonhandicapped 



10. other 

(specify) 
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a) moderately b) mildly c) at risk 

& severely disabled 
disabled 

NONAMBULATORY STUDENTS 

11. mentally 
retarded 



12. learning 
disabled 



13. emotionally 
handicapped 



14. physically 

handicapped 



15. visually 

handicapped 



16. auditorily 
impaired 



17. communication 
disordered 



18. multiply 

handicapped 



19. other 

(specify) 



ZO. other 

(specify) 



Comments: 
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Instructional approaches utilized for students (choose all that 
apply). 

1. task analysis of the skills needed for each job 

2. • individualized instruction 

3. group instruction 

4. lecture 

5. modeling and imitation 

6. peer tutoring 

7. simulations before on-site training 

8. on-site training 

9. reciprocal teaching (students take turns as teachers) 

10. no training 

11. other (specify) 

Comments : 



Type of community or area served by the demonstration project 
(Check one only): 

1. metropolitan area (Public transportation systems) 

2. suburb 

3. small town (5,000-100,000 with no city transportation system) 

4. rural area 

5. entire county 

6. region within a state (including several counties) 

7. statewide 

8. region of nation 

9. nationwide 

10. other (specify) 

Comments : 
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I. Location of students' training (Circle all that apply): 

1. school 

2. workshop/rehabilitation center 

3. "on the job" at a minimum wage job 

4. community 

5. . at home 

6. college or community junior college 

7. other (specify) 

8. no training 

Comments: 



J. Timing of students' training (Choose all that apply): 

1. preplacement 

2. postplacement (on the job) 

3. no training 

4. other (specify)_ 

Comments: 



K. Indicate the number of your students who are in each of the 
following nine employment categories during 1985. 

Employment Category No, of students 

Above minimum wage 

Subsidized (Employer may pay part of wage) 
Part-time 

Full-time 

Not subsidized (Employer pays entire wage) 
Part-time 

hull-time 



Below minimum wage 

Subsidized (Employer may pay part of wage) 
Part-time 
Full-time 

Not subsidized (Employer pays entire wage) 
Part-time 
Full-time 



Not employed 



TOTAL 
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L. Did you place any students in 1984? Approximately (within 

20%) how many students did you place in 1984? 

Comments: 



M. What constrains transition education for your project? 
(Circle all that apply.) 



1. laws constrain placement 

2. funding is limited. 

3. teachers need more skills 

4. housing is hard to find 

5. jobs are hard to find 

6. parents interfere with transition 

7. community attitudes interfere with placement 

8. employers and project staff fear liability suits 

9. other (specify) 

10. other (specify) 

Comments: 



N. What helps transition education for your project? (Check all that 
apply.) 

1. current federal priorities 

2. skilled teachers 

3. workshops in the area 

4. community leader who catalyzes interest 

5. Parent support 

6. other (specify) 

Comments 



0. How did each placed student find "their 11 job? (Circle one) 

1. continuation of school or agency placement 

2. school or agency placement upon school termination 

3. parents 

4. friends 

5. student responded to newspaper ad 

6. placement by an adult placement agency 

7. other (specify) 

Comments: 
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Code Students 

by Number How Job Was Found (Use codes from 

immediately above) 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 
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E. Collect and Review of Initial Annual Progress (evaluation) Reports 

Objective 6.2.1 of the meta analysis was to review and aggregate the 
information from the annual progress reports of OSERS-funded transition 
education projects. The results of this aggregation are reported here. 

Method 

Of the 105 0SER;d projects receiving funding during 1985-1986, 56 had 
original and continuation proposals arrive at the Transition Institute as 
of June 15, 1986. These requests were generally for a third year of fund- 
ing but some were for a second year* 

To determine the effectiveness with which the various transition pro- 
jects are facilitating transition, it seemed best to frame the question in 
terms of whether they were "meeting their contracts": i.e., completing 
objectives that they had reported in their initial proposal for the project 
year being considered. To be consistent across projects, only the project 
year immediately prior to the project year for which funding was being 
requested was examined. Each of the 56 projects with a continuation 
request was examined in six steps. Each step featured a standard that had 
to be met before successive standards could be considered. 
1. Listing of objectives in the original proposal — A listing of major 
objectives in the objectives/goals i n the ori gi nal proposal for the 
project year preceding the project year for which funding was being 
requested was seen as the first step in the accomplishment of these goals, 
a. Only enumerated "objectives" or "goal s" were used as "major 
objectives or goals." "Activities' 1 and "purposes 11 were also used 
in the project proposals, but items under either of these 
headings were not considered "major objectives or goals." 
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b. Objectives or goals that were described and organized vaguely in 
paragraphs so that they were difficult to identify were not con- 
sidered. 

c. If one overall objective or goal with enumerated subcomponents 
was given, each subcomponent was considered to be a "major 
objective or goal" for the remaining items. If more than one 
overall objective or goal were given, the subcomponents were not 
considered to be "major objectives or goals." 

d. If there was (a) no indication of when any of the major objec- 
tives or goals were to be completed or (b) only a listing of 
those to be completed in the first project year for a project 
requesting third year funds, the proposal was searched for an 
activity timeline. An activity timeline was considered an 
acceptable indication of the extent to which the objectives or 
goals were to be completed if it gave at least one activity 
directly related to each objective for the project year preceding 
the project year to be funded. For example, 'printing a bro- 
chure' would have been considered directly linked to a dissemina- 
tion objective and "hiring staff for a second training site" 
would have been considered directly linked to a replication 
objective. 

2 - Listing of objectives in the continuation proposal . In addition 
to an explicit enumeration of the objectives or goals in the original pro- 
posal (Standard #1), the requirement was set that there be a listing of 
major objectives or goals in the continuation proposal for the project 
year preceding the project year for which funding was being requested. 
Points a through d under item 1 were all followed. 
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3. Continuity of objectives from original to continuation pro- 
posal . Correspondence between the two sets of objectives or goals in 
numbers 1 and 2 above was determined by at least one project staff rater. 
Trivial changes (e.g., wording or ordering) were ignored as long as the 
meanings remained in correspondence. Also, the addition of new objectives 
in the continuation proposal did not affect correspondence judgments. 

4. Descri pti on of accompl i shments . The continuation proposal was 
then searched for a description of accomplishments showing clearly which of 
the objectives from step 2 were being discussed. A change in ordering was 
tolerated. Prose presentation of accomplishments without some separation 
(e.g., headings) between objectives was considered to be unclear. 

5. At least one objective reached . The description of accomplish- 
ments was searched for a report indicating that at least one of the objec- 
tives from step 1 had been reached. 

a. Anticipation that an objective would be completed by the end 
of the project year did not count as a report of comple- 
tion of that objective. 

b. For objectives that had subcomponents or an activity list- 
ing, each subcomponent or activity had to be reported as 
completed in order for the objective itself to be rated as 
being reported to be complete. 

c. For objectives involving only initiation of an activity or 
ongoing activity, only initiation or continuation was 
required, respectively, to count them as being reached. 

d. Ongoing objectives needed only to be reported as "occurring- 
to-date 11 to be counted as being reported to be reached. 

6. All objectives reached . The report was checked to see if all of 
the objectives from Standard 1 had been reached. 
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A second person rated the proposals in order to assess the inter- 
rater reliability of the ratings. Proposals were selected ^systemati- 
cally within competitions for these ratings. Proposals were taken randomly 
in turn from each competition until a total of eight proposals were rated 
on standard 6 (above). Prior to this reliability assessment, the two 
raters checked five practice proposals for agreement. These five proposals 
were not used for the reliability analyses reported below. 

Both a percent agreement and a Kappa were calculated to determine the 
reliability of all six standards in succession. The Kappa for each 
successive standard was based on the cases that both raters agreed had 
passed the previous standard. Percent agreement (PA) was calculated by 
dividing the number of agreements by the sum of the agreements and 
disagreements. Kappa was calculated by use of the following formula: 

K = A " E 

where K = Kappa, A = agreements, E = expected agreements by chance, and N = 
number of items (See Cohen, 1960). The Kappa formula corrects for chance 
agreement, which inflates the conventional PA statistic. 

Results 

The number and percentage of proposals meeting each of the standards 
are presented in Table El. 



Insert Table El about here 



A total of 54 project proposals were reviewed. Of this total, 39 
(72%) listed objectives in the original proposal in accordance with Standard 
1. Of the 39 with objectives in the original proposal, 23 {59%) Table El 
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Table El 



Summary of Continuation Proposal Review Findings . 



d % of 



c , . . those 

Standards Number % at step 

0 • P r o j ects for which both an original proposal a54 (1 W%) (100%) 

and a continuation proposal were on file at 

the Transition Institute as of 6/15/86. 

1. Projects reporting objectives for the b 39 72% 72% 
project year prior to that for which funding 

was being requested iji the original 
proposal . 

i 

2. Projects meeting Standard 1 and also 23 42% 59% 
reporting objectives for the project year 

prior to that for which funding was being 
requested in the continuation 
proposal , 

3. Projects for which Standards 1 and 2 are 17 31% 74% 
met and for which the two sets of objectives 

(original versus continuation) correspond 
in meaning. 

4. Projects for which Standards 1 through 3 are c 13 24% 76% 
met and for which accomplishments are listed 

in a fashion such that they clearly indicate 
which of the objectives (from the original 
proposal, as determined in Standard 2) are 
being discussed. 
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Table El (continued) 



reached, 

6. Projects for which all objectives (from the 
original proposal, as determined in 
Standard 2) were reported to have been 
reached. 



d % of 



^ardj Number % Til , 

5. Projects for which at least one objective c 8 15% 62% 

(from the original proposal, as determined 
in Standard 2) was reported to have been 



2% 13% 



A total of 105 proposals were on file at the Transition Institute as 
ot June 15, 1986. However, because those not having both continuation 
and original proposals were excluded from this review, only 56 
projects were available here. Additionally, two of these projects 
were missing the pages on which the original objectives were listed 
?™* W ?- e con sequently excluded from the entire review. Therefore, the 
100% figure is based on only the 54 proposals that were reviewed for 
at least the second standard. 

Two of the projects were excluded for this step (and all subsequent 
steps) as a result of missing pages that made it impossible to 
determine whether they met this standard. 

One project was excluded from step 5 (and all subsequent steps) and 
one was excluded from step 6 because of missing pages that made it 
impossible to determine whether these standards were met. 

Percent of those proposals that passed each step given that they 
had passed the immediately prior standard and were thereby eliqible 
for consideration at the indicated step. 
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listed objectives in the continuation proposal in accordance with Standard 
2. This represented 42% of the 54 total proposals. Of the 23 listing 
objectives in both the original and continuation proposals, 1/ (74%) had a 
correspondence between the two lists. This represented 31% of the 54 total 
proposals. Of the 17 with corresponding lists of objectives in the 
original and continuation proposals, 13 (76%) had a listing of accomplish- 
ments that was clearly linked with the objectives in the original pro- 
posal. This represented 24% of the 54 total proposals. Of the 13 with 
accomplishments that were clearly linked to the objectives in the original 
proposal, 8 (62%) reported completion of at least one objective. This 
represented 15% of the 54 total proposals. Of the 8 that reported reaching 
at least one objective, 1 (13%) reported reaching all objectives. This 
represented 2% of the 54 total proposals. 

The results of the reliability assessment are shown in Table E2. 
Although two of the percent agreements are acceptable (at or above 80%), 
none of the Kappas (which correct for chance levels of responding) even 
approach acceptability (at or above .60). Thus the ratings reported in 
Table E2 are not reliably different from those that would result from a 
random (coin-toss) selection of proposals meeting successive standards. 



Insert Table E2 about here 



Because of the poor inter-rater reliability five proposals were 
selected unsystematically and examined to determine the reasons for the 
disagreements. Based on this review, it appeared that the second rater's 
scoring was more accurate. This rater then reviewed the remainder of the 
proposals, and the second rater's scores were used for the results reported 
above. 
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Table E2 



Results of Reliability Assessment of Continuation Proposal Revi 



ew 



Standard 


% Agreement 9 


b 

Kappa 


N 


2. 


Objectives listed in original 
proposal . 


61% 


.15 


46 


3. 


Objectives listed in continuation 
proposal . 


76% 


.35 


21 


4. 


Correspondence between original 
and continuation objectives. 


79% 


.28 


14 


5. 


Clear linkage between discussion 
of accomplishments and original 
objectives. 


90% 


0 


10 


6. 


Reported completion of at least 
one objective. 


67% 


0 


9 


7. 


Reported completion of all 
objectives. 


83% 


0 


6 



% agreement = agreements/( agreements + disagreements) 
b K _ (agreements - agreements expected by chance) 

(number of cases - agreements expected by chance) 
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Discussion 

Several points are apparent upon examination of Table El. 

1. A surprisingly high number (28%) of the proposals did not list 
objectives for the project year preceding that for which the continuation 
request was being made. Partially this was a result of the fact that most 
of these were requests for third-year funding and some proposal writers 
were reluctant to write specific objectives for the second year in the 
original proposal. However, the absence of objectives in the initial 
proposal made it impossible to determine whether the projects were 
effective in achieving what they had intended to achieve. 

2. The number of projects that listed objectives in their original 
proposal but did not list corresponding objectives in their continuation 
proposal (either because of the absence of an objective list in the 
continuation proposal or a lack of correspondence between the objectives) 
was again surprisingly large (22 of 39). This again hindered determination 
of whether projects met their initial objectives. 

3. Projects with corresponding objective lists in their original and 
continuation proposals usually (76%) listed accomplishments in a way that 
made it fairly easy to identify which objectives were being, discussed. 
However, anecdotally, it had seemed that overall (in all 54 proposals) 
accomplishments were too frequently presented in a format that interfered 
with interpretation (e.g., unordered and intermixed within a single sub- 
section, simply entitled accomplishments). 

4. At first glance it appears that projects often not successful in 
completing their objectives (only 1 of 13 with corresponding objectives and 
clear listings of accompl ishments completed all initially stated objec- 
tives). However, this low completion rate was understandable in view of 
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the fact that continuation proposals were often required after only 2 or 3 
months into the project year for which the objectives were listed. in 
light of this early submission, the fact that 622 of those with 
corresponding objectives and clear listings of accomplishments had 
completed at least one objective is commendable. 

5. Although the reason for the poor reliability obtained for this 
review is unclear, it would appear that proposal guidelines requiring 
Standards 1 through 4 (corresponding listings of objectives in original and 
continuation proposals, and a list of accomplishments in the latter) would 
enhance an outsider's ability to determine what a given project expected to 
achieve and what it in fact did achieve. 

In summary, the projects appear to be progressing toward their objec- 
tives at a rate consistent with their proposed timelines, based on the 
evidence that can be gleaned from the proposals themselves. Two major 
recommendations that would clarify whether progress is occurring at desired 
rates are (a) guidelines requiring standards 1 through 4 and (b) submission 
of progress information at the close of each year of funding. 
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F« Compare Case Studies of Successful and Unsuccessful Job Placements 

This section reports the results to date of the case study meta- 
analysis project. The case study project has been undertaken to identify 
variables that influence success or lack of success in transition from 
secondary school to competitive employment and independent adult living. 

Method 

Each project director was queried about participating in the case 
study project. Those who expressed an interest in doing the case studies 
were sent a packet containing instructions and a "case study checklist." 
Both are shown in Attachment Fl. 

The project director was asked to select one student whose vocational 
placement was successful and one whose placement was unsuccessful. 
"Success" was defined as paid employment for at least 10 hours a week at 
wa g e or better, funded by the employer, and lasting at least six 
months. The subjects selected were to be as similar as possible on the 
variables of age, sex, race, and ability level. The reason for matching 
the subjects on those variables was to control for the influence such 
variables might have on the outcome of the placements. Each case study was 
to be written by someone who knew the subject well. 

The Case Study Checklist was intended to provide guidelines regarding 
the information desired, and to ensure that as many variables as possible 
would be addressed by the respondents. The Checklist has five sections. 
Section A focuses on student characteristics. Section B examines environ- 
mental factors including the home environment, and various characteristics 
of the community. Section C looks at training programs. Section D looks 
at the broader perspectives of support for vocational placements, economic 
incentives, evidence that the student achieved greater independence, and 
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the effects of governmental actions and policies. Finally, section E 
requests a summary of the story of the placement. 

Results 

At the time that this report was prepared, five pairs of students had 
been described In case studies. Each pair consisted of a student whose 
vocational placement was successful and one whose placement was not 
successful. The results are reported below. For ease of comparison, all 
tables are arranged so that the Information for the successful and unsuc- 
cessful students 1n each pair is presented together. Throughout this 
paper, the terms "successful" and "unsuccessful" are used to refer to the 
outcome of the vocational placement and are not intended to reflect aca- 
demic achievement or, indeed, any other areas of their lives. 

Characteristics of the students are shown in Table Fl. 



Table Fl about here 



Table Fl shows the characteristics on which students were matched: 
age, sex, race, and primary disability. It also shows the last grade or 
grade equivalent completed, and each student's school placement history. 
The table shows clearly that the students were well-matched on these 
characteristics. 

Table F2 shows the primary disability and the presence or absence of 
physical or medical conditions that might limit independence. 



67 



Table Fl 

Summary of Student Characteristics 

Age 
Sex 

Ethnicity of home 

Primary Disability 

Last grade completed 

Years in regular class 

Years in regular class 
with support 

Years in special class 
Years in special school 
Years in residential school 

S = Successful 

U = Unsuccessful 

B/I = mixed Black and Indian 

NA = not available 

MR = Mentally Retarded 

MI = Mentally 111 



ERLC 



Pair Number: 



s/u 


1 


2 


3 


4 


5 


s 


18 


22 


19 


19 


22 


u 


18 


22 


20 


19 


20 


s 


M 


M 


M 


M 


F 


u 


M 


M 


M 


M 


F 


s 


B 


W 


W 


B/I 


W 


u 


B 


w 


W 


W 


W 


s 


MR 


MR 


MR 


MR 


MI 


u 


AR 


MR 


MR 


MR 


MI 


s 


11 


12 


12 


12 


12 


u 


11 


12 


12 


12 


12 


c 


o 
C 


n 
U 


1 
1 


MA 


1 


u 


l 


0 


0 


3 


0 


s 


0 


1 


0 


NA 


7 


u 


0 


7 


0 


5 


0 


s 


10 


13 


12 


NA 


0 


u 


13 


8 


9 


3 


2 


s 


1 


0 


0 


NA 


1.5 


u 


0 


0 


2 


1 


2 


s 


0 


0 


0 


NA 


4 


u 


0 


0 


0 


0 


4 
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Table F2 about here 

Three students are reported to have some physical limitations. Occa- 
sional substance abuse by two of the unsuccessful students is reported in 
the case studies. The employment history of the students is shown in Table 
F3. 



Table F3 about here 

The number of placements reported for successful and unsuccessful 
students is similar. Wages, with one exception, were at least the legal 
minimum. All jobs were entry-level positions. The important difference 
for successful and unsuccessful students is the length of time in each 
placement. 

In addition to information about student characteristics, information 
about environmental influences was requested. Table F4 presents a summary 
of information about the students' current living arrangement. 



Table F4 about here 



Five of the students live with their parents, two in apartme;:/ two 
in group homes, and one with his grandmother. Information about the number 
of residents and number of rooms was not provided in some of tha case 
studies. No striking differences between successful and unsuccessful cases 
are noticeable based on the information provided. 
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Table F2 



Physical or Medical 


Conditions That Might Limit Independence 




Successful 


1 


2 


3 


4 


5 


No 


Yes* 


Yes** 


No 


No 


Unsuccessful 


No 


No 


Yes*** 


Q No 


-*m — 



*Mild conductive hearing loss 
♦Limited R arm, weak visual percept. 
*mild spastic cerebral palsy 

Experiments with drugs infrequently 

^occasional alcohol abuse 
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Table F3 



Employment Histories 



Successful Work Placements 



Student Pair 



Job Title 


Hourly 
Wage 


Kitchen Assistant 


3.50 


Dishwasher/Bus boy 


3.35 


Dishwasher 


3.35 


Sandwich Maker/Dish 


3.35 


Dishwasher/bus boy 


4.25 


Janitor 


3.35 


Order f iller/boxmaker 


3.60 


Tire changer 


3.60 


Courtesy clerk 


3.35 


Custodian 


3.35 


Wrap merchandise 


3.35 


Janitorial 


4.00 


Messenger 


4.00 


Maxicare 


4.00 


Microfilm Trainee 


3.85 



Year(s) or 
Length of 
Time Employed 



1 

2 



3 
4 



1 year 
1983 

1983- 85 

1984- 85 

1985- 86 

7 weeks 
10 months 

1 yr. /currently 

1 year 

3 months 
5 months 
3 months 

8 months 

2 months 

1 1/2 months 



Unsuccessful Work Placements 



1 
2 



3 
4 



Pizza preparation 3.35 

Laundry worker 3.60 

Dishwasher 1.08 

Dishwasher 3. 60 

Greenhouse worker 3.35 

Custodian 3.35 
Apprentice Cabinet 

maker 3.35 

Truck Cap Installer 4.00 

Cook 3.35 

Bus boy/prep cook 3.35 

Job Corps na 

Wrap merchandise 3.35 

Food service 3.35 

Food service 3.35 



1985 
9-1983 
1984-85 
6-1985 
2 months 
1 month 



3 months 
1 month 
1 month 
months 
months 
months 
weeks 
weeks 
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Table F4 



Students' Current Living Arrangement 









Pair # 








i 


2 


3 


4 


5 


Living arrangement: 












Successful 


Relative 


GH a 


Parents 


Parents 


Qii noru" 

Apt. 


Unsuccessful 


Parents 


GH 


Parents 


Parents 


Own 
Apt. 


# of Residents: 












Successful 


2 


8 


? 


5 


? 


Unsuccessful 


4 


8 


? 


5 


1 


# of Rooms: 












Successful 


3 


? 


? 


8 


? 
• 


Unsuccessful 


6 


? 


? 


10 





*Note: GH = Group Home 
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Table F5 summarizes reports of the quality of the neighborhood in 
which the student lives. it appears that successful and unsuccessful 
students live in similar neighborhoods. 



Table F5 about here 



Table F6 compares the training programs of the successful and 
unsuccessful students. The intensity and thoroughness of the training 
programs for these students is striking. 



Table F6 about here 



In only one instance did the training programs differ. The unsuccessful 
student in pair 3 received training in daily living skills while the 
successful student did not. 

Table F7 focuses on the decision-making influence of various persons 
or teams regarding the total training and placement program for each 
student. A total of 100 "influence points" were divided among the individ- 
uals or groups listed as "Sources of Influence." 



Table F7 about here 



Differences of 5 to 10 points in the reported influence of any person or 
group on the training and placement of the successful and unsuccessful 
students can be seen. There is no pattern that might suggest that greater 
or lesser amounts of input from any source affects placement outcomes. 
Table F5 
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Table F5 



Quality of Neighborhood 



56 



"ritzy- 
professionals 
middle class 
apartments 
blue collar 
mixed commercial 
slum 
other 



U 
B 
U 



Pair Numbers 
2 3 4 



U 
U 

S B B 

S 

B 

U 



This information was not provided for pair #5. 

3 Note: B = Both students 

S = Successful student 
U = Unsuccessful student 
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Table F6 

Characteristics of Training Program 

Pair Number 





1 


2 


3 


4 


5 


personal/social skills 


a B 


B 


B 


B 


B 


daily living skills 


B 




u 




B 


career guidance & planning 


B 


B 


B 


B 


B 


academic skills 


B 


B 




B 


B 


travel skills 


B 






B 


B 


job readiness skills 


B 


B 


B 


B 


B 


job placement 


B 


B 


B 


B 


B 


follow-up, long term support 
and supervision 


B 


B 


B 


B 


B 


other 






B 


B 




other 






B 


B 





Note: B = Both students 

S - Only successful student 

U = Only unsuccessful student 

- = Neither 
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Table F7 



Influences in Finding Placements 

Sources of Influence a S/U Pair Number 







1 


2 


3 


4 


5 


Intake assessment & evaluation 


S 


10 


10 


0 


5 


10 




■ i 

u 


10 


5 


0 


5 


10 


Preplacement trainers and/or 














teachers 


S 


30 


20 


20 


25 


5 




u 


30 


15 


40 


20 


5 


Placement counselors 


s 


10 


5 


15 


20 


5 




u 


10 


5 


15 


25 


5 


uj I irainers and/or teachers 


s 


20 


20 


15 


20 


30 




u 


20 


15 


15 


20 


30 


bupervisors ot direct service staff 


s 


0 


5 


0 


0 


30 




1 1 

u 


0 


10 


0 


0 


30 


Agency Board 


s 


0 


0 


0 


0 


0 




u 


0 


0 


0 


0 


0 


Parents and other family 


s 


20 


5 


20 


25 


10 




u 


30 


15 


10 


25 


10 


Friends and advocates 


s 


10 


0 


0 


5 


10 




u 


0 


0 


0 


5 


10 


Other 


s 


0 


15 


5 


0 


0 




u 


0 


20 


20 


0 


0 


Other 


s 


0 


20 


0 


0 


0 




u 


0 


15 


0 


0 


0 



Note: S = Successful placement 
U = Unsuccessful placement 
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Table F8 shows sources of support for the student's placement and how 
much each is believed to have contributed in support of the placement. A 
total of 100 "support points" was distributed among the categories listed. 



Table F8 about here 



If there are any trends, they are very subtle. Students' superior ability 
and more extensive on-the-job supervision were given more credit for sup- 
porting the placement of the successful student than of the unsuccessful 
student. "Enlightened leadership in transition agency," and "follow-up 
from transition agency" were reported as more common for unsuccessful 
students. 

Table F9 describes economic incentives and disincentives regarding 
placement. Very few incentives or disincentives are reported. Students 
earn wages, pay taxes, and lose SSI. 



Table F9 about here 



Table F10 shows who pays students during the evaluation, training, and 
placement process. 



Table F10 about here 



Departments of Rehabilitative Services (DRS) provided funding for four 
students. Evidence that the placement helped the student achieve increased 
independence was requested. Table Fll summarizes the evidence. 
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Table F8 



Source and Importance of Job Placements 



Sources of Support 


S/U 


1 


Pan r 

2 


Number 
3 


4 


5 


student 1 s abil i ty 


s 


20 


20 


15 


50 


30 




u 


20 


10 


20 


5 


0 


peer(s) on the job 


S 


20 


10 


0 


0 


0 




u 


0 


20 


0 


0 


0 


supervisors on the job 


s 


20 


20 


30 


5 


20 




u 


10 


5 


20 


5 


30 


follow-up from transition agency 


s 


20 


0 


15 


30 


30 




u 


30 


10 


30 


50 


0 


family support 


s 


20 


5 


30 


10 


0 




u 


10 


15 


10 


25 


20 


good judgment in student placement 














matchup 


s 


0 


20 


10 


5 


30 




u 


10 


5 


20 


5 


0 


luck 


s 


0 


10 


0 


0 


0 




u 


0 


10 


0 


0 


0 


enlightened leadership in job 














nl Aromonf cottinn 
p 1 uUcINcil L ocLLIiiy 


c 
o 


n 
u 


L 0 


n 
u 


n 
u 


n 
U 




u 


0 


0 


0 


5 


0 


enlightened leadership in 














transition agency 


s 


0 


0 


0 


0 


20 




u 


20 


0 


0 


5 


50 


other 


s 


0 


0 


0 


0 


0 




u 


0 


25 


0 


0 


0 


other 


s 


0 


0 


0 


0 


0 




u 


0 


0 


0 


0 


0 
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Table F9 



Economic Incentives and Disincentives to Job Placempnt 



Economic 
incentives 
or dis- 
incentives 
for: 

Student 



Training 
Staff 

Training 
Agency 



Placement 
Site 



Funding 
Agency 

Taxpayers 
Others 



S 
U 



S 

u 
s 
u 
s 
u 



s 
u 

s 
u 

s 
u 



1 

wages loss 
wages, job, loss 
trainer 

None reported None 
None reported None 

funding based None 
on placements 
"Yes" None 



Pair Number: 



of SSI 
of SSI' 



reported 
reported 

reported 

reported 



None reported TJTC 

TJTC, None reported 

trainer, 

stipend 

paid 

None reported None reported 
None reported None reported 

None reported Yes 
None reported Yes 

None reported None reported 
None reported None reported 



3 A c 


NA 
NA 


None reported 
None reported 


NA 
NA 


NA 
NA 


None reported 
None reported 


NA 
NA 


NA 


None reported 


NA 


NA 


None reported 


NA 


NA 

TJTC 


Cash Incentive 
for placement 
Employee 


NA 
NA 


MA 
NA 


None reported 
None reported 


NA 
NA 


NA 
NA 


S Is paying taxes 
S Is paying taxes 


NA 
NA 


NA 
NA 


None reported 
None reported 


NA 
NA 



S - Successful, U » 
TJTC - Targeted Jobs 



Unsuccessful, NA • Not answered, 
Tax Credit 
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Table F10 



Sources 


of Funding for Evaluation, Training, Placement 


Pair 
Number 


Successful 


Unsuccessful 


1 


Employer Project Workability 
(State Dept. of Education) 


Project Workability 
stipend 


2 


Employer DRS (OJT program) 


DRS (OJT money) 


3 


Incentive from Dept. of Rehab. 
Services (DRS) 


Incentive 
from DRS 


4 


Employer 


Employer 


5 


NA 


NA 



NA = Not answered 

DRS = Department (or Division) of Rehabilitation Services 



Table Fll about here 



This information was not provided for one pair of case studies. 
Evidence was reported that the other four of the successful students had 
gained increased independence in the four areas of concern. Unsuccessful 
students showed evidence of increased independence in only two or three 
areas. 

Discussion 

The successful and unsuccessful students in the case study project 
were well matched on a number of variables that one would expect to expose 
their differences. They were remarkably similar in personal character- 
istics , environmental experiences, and training programs. The projects 
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selected well-matched subjects for their case studies. Successful and 
unsuccessful students in each pair differed only slightly on the personal 
characteristics identified in Tables 1 and 2: age, sex, ethnicity, 
disability, school placement history, and physical or medical conditions. 
The current living arrangements, shown in Table F4, were similar, and the 
quality of the neighborhood in which the students lived, shown in Table 5, 
was also very similar for each pair. The successful and unsuccessful 
students in each pair work in communities whose economies are based on the 
same industries, as shown in Table F6. Table F5 shows that attitudes 
toward handicapped individuals are similar in the cormiunities where the 
successful and unsuccessful students live. This finding is, of course, 
partially spurious, since both members of most pairs presumably came from 
the same community. Reported economic incentives and disincentives (Table 
F9), and funding to pay student during evaluation, training and placement, 
(Table 12) are similar for both successful and unsuccessful students. 

Subtle differences are noted between successful and unsuccessful 
students in Tables F3, F6, F7, and Fll. Table F3 shows, as would be 
expected, the unsuccessful students were employed for a much shorter time 
than were the successful students. Differences in the type of job and 
wages were not noted, except that one unsuccessful student held a job at 
the substandard wage of $1.08 per hour. 

In Table F7, differences in the influence attributed to various 
sources varies from 5 to 10 points, and in one instance by 20 points. But 
for most of the persons on teams listed, the same amount of influence is 
attributed whether the outcome is successful or unsuccessful. 

Greater differences are noted in the sources of support, shown in 
Table F8. Student ability and on-the-job supervisors seem important when 
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Table Fll 



Evidenced of Increased Independence 





S/U 


1 


2 


3 


4 


5 


Educational evidence 


S 


Yes 


Yes 


NA 


Yes 


Yes 




U 


No 


Yes 


NA 


Yes 


Yes 


Vocational evidence 


S 


Yes 


Yes 


NA 


Yes 


Yes 




U 


Yes 


Yes 


NA 


Yes 


No 


Living skills evidence 


S 


Yes 


Yes 


NA 


Yes 


Yes 




U 


Yes 


Yes 


NA 


Yes 


No 


Behavior change 


S 


Yes 


None 


NA 


Yes 


Yes 








needed 










U 


Yes 


None 


NA 


No 


No 



needed 



S = Successful Student 
U = Unsuccessful Student 
NA = Not answered 
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the student is successful. The transition agency appears important when 
the student is unsuccessful. 

In Table Fll successful students were seen by their agencies to have 
become more independent than were the unsuccessful students. 

The differences between successful and unsuccessful stvwfents that have 
appeared in these tables are suggestive but mt coyiclustve. The small 
number of responses to date makes it impossible to kru>w which , if any, of 
the differences is meaningful. When the total complement of 50 cases is 
completed, summary statements can be made with more certainty. 
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University of Illinois 
at Urbana-Champaign 



Secondary Transition Intervention 
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1 10 Education Building 
1310 South Sixth Street 
Champaign, Illinois 61820 



CoQege of Education 



(217) 333-2325 

(217)333-0260 
(Heal) 
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TRANSITION INSTITUTE 
CASE STUDY META-ANALYSIS 
INSTRUCTIONS AND FORMAT 
Laird W. Heal 
April 17, 1986 

The Secondary/Transition Institute at the University of Illinois has 
been charged by the Office of Special Education and Rehabilitative 
Services to identify the factors that lead to successful transition 
education. This case study project will attempt to identify these 
factors. 

We are asking each model transition project funded by 0SERS to help 
us by providing two case studies: one of a successful placement (i.e., 
paid employment tor at least 10 hours per week at minimum wage or better 
funded by the employer and lasting at least six months) and one of an 
unsuccessful placement (i.e., one that lasts less than six months). The 
successful and unsuccessful cases should be matched as closely as possible 
on their personal characteristics (ability profile, age, sex, ethnicity, 
motivation, handicap profile* etc.) so that success of placement can be 

attributed to environmental and program characteristics rather than to 

personal characteristics of the student. 

My staff and I will review these case studies and combine the 

information from about 50 different pairs of students in order to identify 

the features of placement and training that appear to distinguish between 

su :essful and unsuccessful placements. 
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Please tell the story of the student that you have chosen as 
accurately as you can. You have been selected to write this story because 
you were recognized to have special insights into this student and the 
targeted job. Your opinions and judgments are, therefore, at least as 
valuable as your knowledge of the raw facts. 

In order to assist my staff and me in combining the information from 
so many case studies, we are asking that everyone follow the same 
outline. Please cover the topics listed below using the numbering system 
that has been provided to identify the sections of your report: 
(a) Please complete all checklist items, making liberal comments, adding 
new choices, or modifying our choices in order to make your responses as 
accurate as possible, and (b) write a narrative about, the case in as much, 
or as little, detail as you please. A major purpose of the narrative is 
to clarify checklist responses. But iven more important is to describe 
important aspects of the placement that the checklist does not request. 

Some of the information requested by the Case Study Checklist below 
may be difficult for you to obtain. While we would like to have complete 
information on every student, we are much more interested in reading your 
story than we are in getting every item completed on the Checklist. 

Our research guidelines require that the anonymity of each 
student be protected, and that no participant be coerced to participate 
in research involuntarily. Therefore, please use fictitious r its for any 
student you choose to stujy, and tell them of their right to refuse to 
answer any questions that you might want to ask them. It is not 
considered unethical to consult project files in order to tell the story 
of an individual student as long as the student's anonymity is preserved. 
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We believe that this project will make *: -"..-jortant contribution to 
our knowledge of successful transition educati and we appreciate your 
important contribution to it. 

Laird W. Heal 
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University Of Illinois Secondary Transition Intti wmn ColJcge of Education 

Effectiveness Institute 



at Urbana-Champaign 



! !0 Education Building (217)333-2325 70 

1310 South Sixth Street 

Champaign, Illinois 61820 (217)333-0260 

(Heal) 



CASE STUDY CHECKLIST 
Laird W. Heal 
April 17, 1986 

Student characteristics and background. 

1. Student's age. 

2. Student's sex. 

3. Ethnicity of the student's home. 

4. Last grade completed or equivalent. 

5. School placements since kindergarten. (Enter year(s) and checks 
in the spaces below.) 

Year(s) Regular Regular Special Spedcrl Residential 
Class Class & Class School School 
Support 



7. List all jobs held and the hourly wages of each job. 

Year(s) Job Title or DOT Code Hourly Wage 
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8. How is this student typical of those of a similar age, sex, and 
overall ability, and how is he or she different? Discuss 
special talents or disabilities, special family or friendship 
situations, or job characteristics that might have affected this 
student's placement, making it better or worse than that of his 
peers. 

9. Does this student have any physical or medical conditions that 
might limit his or her independence? List physical conditions 
as well as regular or occasional medications. 

Environment and environmental history. 

1. Describe the student's home environment. Include significant 
information about: 

a. Parents, guardians, supervisors 

b. Siblings 

c* Number of residents 

d. Number of rooms 

e. Status, privileges, and responsibilities of the student in 
his or her home 

2. Quality of neighborhood (Choose all that appl>.) 

a. "ritzy" 

b. professionals 

c. middle class 

d. apartments 

e. blue collar 

f. mixed commercial 

g. slum 

h. other (describe) 
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3. Describe the home environment of this student from birth to the 
present time, stressing the factors that seem to have most 
influenced his or her vocational development. How was the 
student regarded by parents, siblings, peers, and neighbors as 
he or she was an infant, toddler, pre-schooler, and student. 
Please be brief, stressing the student's preparation for 
employment. 

4. Describe the economy of the community where the student is (was) 
employed. Which of the following industries appear to be more 
prevalent or less prevalent than they would be in the average 
community? Please check one choice for each industry: 
Industry Less prevalent Average More prevalent 

v. 

Agriculture 

Manufacturing 

Retail trade 

Wholesale trade 

Government 

Service industries 

Construction 

Education 

5. Describe the attitude of this community toward handicapped 
students and their future life styles. Discuss vocations, 
residences, and leisure activities. Are there ample job 
opportunities for handicapped people? Are handicapped people 
integrated into regular apartments? Are handicapped individuals 
often seen in public? Focus on the features of the community 
that directly influence the student. 
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6. Describe the human service system that can be expected to 
support the vocational, residential, and leisure status of the 
student. If feasible, draw an organization chart showing the 
governing and cooperative relationships among the agencies 
responsible for this student's vocational, residential, and 
residential services. 

Program characteristics 

1. Our primary descriptions of your program will be the "profile" 
prepared by the Transition Institute Evaluation faculty under 
the direction of Drs. Allen Phelps and Lizanne DeStephano, and 
the project abstracts that appeared in your grant proposals. If 
you feel that these documents need expansion, please supplement 
them- 

2. Describe the training program for this student. Which of the 
following program characteristics apply to the programming for 
this student? (Check all that apply.) 



a. 


Personal and social skills 


b. 


Daily living skills 


c. 


Career and guidance planning 


d. 


Academic skills 


e. 


Travel skills 


f. 


Job readiness skills 


g. 


Job placement 


h. 


Follow-up and long-term support and supervision. 


i . 


Other (specify) 


J. 


Other (specify) 
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Estimate the decision-making influence of each of the following 
teams or individuals regarding the total training and placement 
program of this student. In making this estimate, assign a 
total of 100 "influence points" to the following 11 "actors" in 
the placement process. 



1 ill 1 UCIIL6 


1 earn or i nu 1 v i aua i 


Pni nt^ 

r \j i ii to 






Tntakp a^^p^^mpnt" anH pval na tinn tpam 

JlllLiUNC aodC03IIICll t UlIU CVQ 1 UQ u IUII LCulll 


b. 


Pre-placement trainers and/or teachers 


c. 


Placement counselors 


d. 


On-the-job trainers and/or teachers 


e. 


Supervisors of direct service staff 


f. 


Agency Board 


g. 


Parents and other family 


h. 


Friends and advocates 


i . 


Other (specify) 


j. 


Othe^ (specify) 


100 


Total (The sum of all points must equal 100) 
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Some broad perspectives 

1. Assign "support points 11 to each of the following supports for 
the student's placement. What has kept this student on the job 
OR, in the case of an unsuccessful placement, what profile of 
energies was focused on the effort to keep this student on the 
job? Please assign 100 effort points to the U support 
categories. 



Support Sources of support 

Points 

a. student's ability 

b. peer(s) on the job 

c. supervisor(s) on the job 

d. follow-up from transition agency 

e. family support 

f, good judgment in student-placement match-up 

g. luck . 

h. enlightened leadership in job placement setting 

i. enlightened leadership in transi**"" agency 

j. Other (specify) 

k. Other (specify) 

100 (The sum of all points must be 100) 
2. What economic incentives or disincentives regarding placement 
are/were there for each of the following individuals or groups 
involved in this placement? 

a. the student (e.g., loss of SSI) 

b. the training staff (e.g., bonus for each placement) 
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c. the training agency (e.g., funding based on the number of 
students placed) 

d. the placement site (e.g., free advertising) 

e, the funding agency(s) (e.g., increased appropriations) 

f. taxpayers (e.g., removal of client from SSI) 

g, other (specify) 

h, other (specify) 

3. Who pays the student throughout the evaluation, training, and 
placement process? 

4. What evidence is there that the student's program has helped him 
or her to achieve greater independence? 

a. educational evidence (e.g., improved from no coin 
recognition to knowing labels for all U.S. coins) 

b. vocational evidence (e.g., improved from no knowledge of 
dishwashers to independent operation) 

c. living skills evidence (e.g., moved from ICFDD to apartment) 

d. evidence of reduction of destructive, disruptive, or 
inappropriate behavior (e.g. , swearing rate was reduced 
from e word in 10 to one in 200) 

5. What actions and/or policies of local, state, and federal 
governments facilitated or impeded the education, and placement 
process of this student. 

Summary. Tell the story of this placement in a nutshell. Write one 
or two sentences completing each of the following items. 

1. John is . . 

2. John grew up . 




John now lives 

John was selected for placement at 

because (skills, employer compatibility, training;. 

On the job, John 

The benefits of the placement have been 

The difficulties of the placement have been 

The placement has (succeeded, failed) because 
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G - Compare OSE RS Research Projects on Transition Education 

At the annual meeting in October, 1985, in Washington, D.C., the staff 
of the meta-analysis evaluation sub-component was urged to contact the 
projects that had been funded to do research on transition education. 

Accordingly, objective 6.2.12-Compare OSERS research projects from 
competitions 84.158C (7 projects) and 84.158G (15 projects)-was added to 
the management plan from the original proposal when objective 6.2.3 
(Develop a framework for the meta-analysi*) was conpleted in March, 1986. 



Method 



A one-page questionnaire (Attachment Gl) was completed for all 22 
research projects from the two competitions. This was mailed to the 
84.158G competition and completed by Institute staff for the 84.158C 
competition. Ten of the 22 questionnaires were completed by the 84.158G 
project staff, 6 were completed from telephone interviews with project 
staff, and 6 were complied solely on the basis of project abstracts. No 
reliability analysis was done. 



Table Gl about here 



Results 

The results of the survey of the OSERS research competitions 84.158C 
and 84.158G appear in Table Gl. Section A of this table shows that only 16 
of the 22 projects directly served handicapped transition education stu- 
dents. Furthermore, (Section B) only seven of these placed students, 
although section D shows that an additional 5 who placed, trained, or 
assessed students in placements made by other agencies. Section C shows 
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that at least 12 of the 22 projects were documenting their research proce- 
dures, and only three have no apparent plan to do so. 

Finally, section D shows the most crucial data from the survey: the 
cross-tabulation of the type of experimental design by the three degrees of 
involvement in the placement of students into competitive jobs. As noted 
above, only 12 projects (sum of columns 1 and 2) had interventions that 
directly affected student placement. Also, it is clear that none of these 
had randomly established control groups and only four had control groups of 
any kind. 

Conclusion 

The Transition Education service projects funded by OSERS do not lend 
themselves to a conventional meta-analysis. A conventional meta-analysis 
uses the strategy of creating a standardized measure of the difference 
between the success of treated and untreated subjects, that is, between 
experimental and control groups. Because only four of the research 
projects funded by OSERS employed control groups (and none had the pre- 
ferred randomly assigned control group), there is little point in engaging 
in a conventional meta-analysis, which should have at least five times as 
many projects as predictor variables. Thus, objective 6.2.12 will end with 
this report. 
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Table Gl 

Results for the Transition Education Feasibility Survey 

Funded projects from OSERS 
Competitions 84.158C and 84.158G 

A. Population Served 

Number of Projects 



Ifi 


Developmental!,/ Disabled Students within the 
13-26 age range 




_6 


None, indeterminate, not applicable 




22 


1 Ota 1 




B. Project activities 






Number of Projects 


Activity 




10 


Student or client pre-service assessment 




15 


Student or client training 




7 


Student or client work placement 




8 


On-the-job training of clients or students. 
Training after placement. 




8 


Student evaluation after placement 




C. Manual to document project 




Number of projects 


Activity 




9 


Manual exists 




3 


Manual in preparation 




3 


Exact procedures not to be documented 




7 


Unknown 




22 


TOTAL 
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Table 61 (continued) 



D. Experimental design for projects who placed or did not place stu dents 



Placement status of project 



Design 



Placements 
Placements are followed Placements 
are made but not made are not made 



2 
4 



2 
2 



3 
6 



16 



True experiment (Random 
assignment to experimental 
and control groups 

T ntact groups (No random 
assignment to groups) 

Multiple baseline: time 
series with intra-subject 
comparisons 

Single jroup of students 
with no non-intervention 
condition 

Total 
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Attachment Gl 

Transition Education Summary Analysis Feasibility Questionnaire 
Laird W. Heal - February 10, 1986 
1. Official Title of Project: 



2. Project Director: 

3. Mailing Address: 



4. Telephone: 



0 1 9, 



5. This project is funded by 

6. Students or clients served directly: No = 0; Yes = 1 



a 



0 1 a. Young adults from 13-2C 
0 1 b. Developmentally disabled 

7. Project activity: No = 0; Yes = 1 

0 1 a. Student or client pre-service assessment 
0 1 b. Student or client training 
0 1 c. Student or client work placement 
Old. On-the-job training of clients or students 
0 1 e. Student evaluation after work placement 

8. Are the procedures used for the project's activities written down 

in a manual? 



Does your project define a successful placement in some objective way? 



0 i 10. HOW? 



Do you have an experimental or quasi-experimental design to 
determine whether your project's procedures are an improvement upon 
the transition education program that would exist in your absence. 
When would you respond to a request for your procedures and 
successful placements mentioned in items 7-9. 



Note: Circle "0" to indicate a "No" response or "1" a "Yes" response. 
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H. TASK 6.2: A META-ANALYSIS OF TRANSITION EDUCATION: Year 2 Activities 

August 21, 1986 - August 20, 1987 
Graduate Assistant: Janell Haney 

A meta-analysis is an aggregation of information from a number of primary 
sources. As applied in the present series, it will be an aggregation of 
information about successful and unsuccessful transition from secondary 
education to post-secondary settings— information provided by the projects 
that are being funded by the Office of Special Education and Rehabilitative 
Services. 

Two strategies will be applied. The first will be a conventional attempt 
to identify the variables associated with high rates of successful 
placement (Objective 6.2.4 and 6.2.13). Projects will be classified 
according to their goals and procedures. Variations in classification will 
form the independent variables of this analysis. Placement outcomes for 
each project will be measured. These might include proportion of graduates 
placed in part- or full-time employment, average wages of graduates, 
proportion of students placed into cooperative part-time employment, 
per-student program cost, etc. This analysis strategy will program 
characteristics that are associated with high rates of quality placements. 
The second meta-analysis approach will be the aggregation of information 
from a number of successful and unsuccessful placement cases (Objective 
6.2.11). OSERS service projects are asked to nominate successful and 
unsuccessful placements of students from high school training programs into 
the competitive employment marketplace. Each project has been asked to 
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nominate a typical successful case and a typical unsuccessful case. The 
reasons for success or failure will be noted in a case study that will be 
completed according to a structured format. This format will yield both 
objective and subjective information about these cases. Here the 
individual will be the unit of analysis, and the effort will be to identify 
individual characteristics, training programs, and placement situations 
that maximize placement quality and minimize placement failure. 

The first year of the project has focused on sharpening the measurement 
instruments and instructions to individual projects and identifying the 
data that are widely available over many projects. A preliminary report of 
this work, including tentative findings, will appear in the first annual 
report. The analysis and reporting of this pilot work will be completed by 
January of 1987. in the meantime, instruments and instructions will be 
fine-tuned, and both meta-analyses will be repeated in the winter of 1987. 
A final report of these meta-analyses should be ready for the second annual 
report in the summer of 1987. 



It is difficult to see beyond the summer of 1987, but it would probably be 
valuable to continue to aggregate data, realizing the cooperating service 
projects will improve their data recording procedures year by year, and the 
institute will improve its instruments, instructions, and analyses. 



Procedures 



6.2.13 (Year 2 of 6.2.4 - Meta-analysis by Project Groups). 
Responses from the questionnaire mailed in the last week of May, 1986, 
will be tabulated and analyzed using the IBM statistical package, 
SAS. Multivariate least squares statistical procedures will be used 
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to indicate the project characteristics that appear to be related to 
high rates of quality placements. These results will be written in a 
prel imi nary report for the I ns ti tute 1 s annual meeti ng i n October , 
1987. After review by the Institute Advisory Board and the OSERS 
project officer, a final report will be completed and disseminated in 
January, 1987. The questionnaire will be revised and the projects 
will be asked to complete it in the early spring of 1987. Sufficient 
numbers shoul d be returned by June , 1987 , to al 1 ow a prel imi nary 
report of the results to be included in the 1987 annual report. 

6 .2.12. (Meta-analysis of Case Studies) . Responses from the Case 
Study Guidelines mailed to projects on April 21, 1986, will be 
analyzed using a content analysis coding system to be developed by 
project staff in the summer of 1986. Multivariate least squares 
statistical procedures from the IBM SAS statistical package will be 
used to indicate which personal, employment, training, and supervisor 
characteristics appear to be associated with successful and unsuc- 
cessful job placement. A preliminary report will be prepared for the 
annual meeting in October, 1987. After review by the Institute 
Advisory Board and the OSERS project officer, a final report will be 
completed and disseminated in February, 1987. The Case Study guide- 
1 i nes wil 1 be amended and projects wi 1 1 be asked to conti nue to 
complete case studies on matched pairs of successfully and unsuc- 
cessfully placed pairs of handicapped students. New cases will be 
added to the analyses and a final report of all aggregated cases will 
be prepared for the annual report in July, 1987. 
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Products 

The products will be two major reports, each with one or two 
preliminary drafts. Each one will be submitted for publication in a 
scholarly journal. The production schedule is detailed as In the 
management plan and timetable Immediately below. 
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